Runtime Stealthy Perception Attacks against DNN-based Adaptive Cruise Control Systems

Introduction

This slide introduces the core threat explored in the paper: stealthy runtime adversarial attacks on vision-based Adaptive Cruise Control (ACC) systems. It highlights how small, targeted image perturbations can mislead the deep neural network into misjudging the position of the lead vehicle, creating a discrepancy between the actual and predicted positions. The attacker subtly alters the live camera feed, fooling the system into thinking the path ahead is clear—leading to a failure to brake and potentially resulting in collisions. The visual flow on the left illustrates how adversarial inputs (e.g., noise added to a stop sign) can propagate through perception models to cause dangerous misinterpretations. This sets the stage for examining how such attacks work, why they are hard to detect, and what implications they have for the safety of real-world autonomous driving systems.

Introduction 2

This slide provides a visual introduction to how Adaptive Cruise Control (ACC) works in modern vehicles. ACC enhances traditional cruise control by dynamically adjusting the vehicle’s speed based on real-time data from sensors such as cameras, radar, or LIDAR. As shown, the system first sets a cruising speed, then uses perception inputs to detect vehicles ahead. If a lead vehicle is detected, ACC automatically reduces speed to maintain a safe distance. Once the lane is clear, it resumes the preset cruising speed. At the heart of ACC functionality is accurate detection and tracking of the lead vehicle, which makes the system highly dependent on the reliability of its sensor-based perception pipeline.

Motivation

This slide presents the motivation behind studying adversarial attacks on ACC systems. It underscores the critical reliance on object detection and tracking for safe operation—any failure in perception directly impacts the vehicle’s ability to maintain safe distances. Existing attack strategies often rely on offline optimizations, making them easier for humans to detect or mitigate. Moreover, built-in safety interventions and anomaly detection mechanisms can sometimes flag suspicious behavior. However, as the image illustrates, even subtle changes like an adversarial patch on a vehicle can fool the model into misclassifying or mislocalizing key objects, all while appearing benign to a human observer. This raises concerns about how easily attackers can exploit the visual pipeline without triggering standard safeguards.

Aim and Contributions

This slide outlines the aim and contributions of the study. The central objective is to examine the vulnerability of ACC systems in runtime conditions where a human is still in the control loop—a key aspect of Level-2 autonomous vehicles. The research focuses on identifying the optimal timing and scenario for launching adversarial attacks that can realistically result in collisions. To achieve this, the authors develop a dynamic optimization-based attack strategy that operates in real-time, rather than relying on static, precomputed plans. As shown in the architecture diagram, the attack targets the DNN-based perception module that feeds into the longitudinal planner of the ACC system. The study further validates its findings through both simulation and real-world testing, with special attention to safety interventions and human oversight.

Adaptive Cruise Control

This slide breaks down the internal structure of an Adaptive Cruise Control (ACC) system, emphasizing how multiple components interact to ensure safe driving. The system relies on a range of sensors—including cameras, radar, IMU, GPS, and LIDAR—to detect and monitor surrounding vehicles. A deep neural network (DNN) processes this sensor data to estimate relative distance (RD) and relative speed (RS) to the lead vehicle. These values feed into a longitudinal planner, which determines the required acceleration, deceleration, or braking to maintain safe following distances. The vehicle control unit then executes these actions to minimize risk while transitioning to the next state 𝑠 𝑡 + 1 s t+1 . This layered architecture highlights the importance of reliable perception, making it a prime target for adversarial manipulation.

Attack Model

This slide defines the attack model explored in the study, focusing specifically on manipulating the input to the deep neural network (DNN) to achieve stealthy interference. The attacker’s primary constraint is the ability to modify live camera feeds in real-time. To carry out such an attack, the adversary is assumed to possess a certain level of knowledge about the ACC system design and access mechanisms. These include intercepting image frames during runtime via methods like over-the-air updates, remote access, or even physical projection attacks. The accompanying threat model table categorizes attacker strength—ranging from strong (e.g., malware with internal access) to weak (e.g., external physical manipulations)—along with the corresponding system impact. The focus on runtime DNN input manipulation allows attacks to remain subtle and bypass traditional detection mechanisms.

Attack Challenges

This slide highlights the key challenges in launching successful runtime attacks on ACC systems. The first challenge (C1) is determining the optimal timing for an attack—such as when the lead vehicle (LV) is present but undetected—maximizing the risk of a collision. The second challenge (C2) involves generating effective perturbations in real time that adapt to constantly changing driving conditions. Offline-generated attacks often assume fixed vehicle types or scenes, which don’t generalize well in practice. Lastly, challenge C3 addresses the tight real-time constraints of the system; any attack must be computed and executed before the next frame is processed, leaving only milliseconds for optimization. The images visually depict a successful attack where the lead vehicle disappears, misleading the ACC system into thinking the road is clear.

Attack Design

This slide illustrates the design of the runtime attack targeting ACC systems. The attack pipeline is divided into offline and runtime stages. In the offline phase, attackers use reverse engineering and credential acquisition (e.g., SSH keys) to gain system access. During runtime, the attack leverages context-aware activation—monitoring live camera frames to determine the optimal moment for interference. It then uses adaptive adversarial patch generation, injecting subtle perturbations or projections into the input stream to fool the DNN model. The left diagram visualizes this multi-stage process, while the safety context table on the right shows how even minor misjudgments—such as accelerating when the headway time (HWT) is unsafe and the relative speed (RS) is positive—can lead to dangerous outcomes. This demonstrates how an attacker can exploit perception errors to violate predefined safety rules and provoke collisions.

Attach Design 2

This slide dives deeper into the mathematical formulation and implementation of the attack design. The goal is to generate an adversarial patch Patch 𝑡 Patch t that perturbs the camera frame in a targeted and constrained way, maximizing the DNN’s error in estimating the relative distance (RD) and relative speed (RS) of the lead vehicle. The optimization balances stealth (via small patch norms) and effectiveness (gradient-based loss maximization) under physical and perceptual constraints. The diagram illustrates the full pipeline: clean camera input is subtly modified with a patch generated by identifying key visual regions (e.g., the rear of the lead vehicle), which are then processed through the DNN. The altered perception misleads the control logic of the ACC, ultimately impacting throttle and brake decisions. This approach allows real-time, scene-adaptive attacks that remain hidden from both human drivers and safety systems.

Safety Intervention Simulation

This slide explains how the study evaluates attack robustness in the presence of safety interventions. Using a simulation setup involving OpenPilot, CARLA simulator, and a virtual driver module, the system models how both automated responses and human drivers react to hazards. The framework supports three configurations: (1) AEBS (Automatic Emergency Braking System) enabled with intact camera input, (2) AEBS enabled but compromised by the attack, and (3) AEBS disabled entirely. The driver simulator table details the conditions that trigger responses, such as unexpected acceleration or unsafe following distances, with typical reaction times set at 2.5 seconds. The diagrams illustrate how alerts are prioritized: AEBS acts first, followed by the driver, and finally the ACC system. This layered design allows the researchers to test whether attacks can evade or overpower safety mechanisms under different real-world scenarios.

Simulation Methods and Results

This slide presents the simulation setup and key experimental results addressing three core research questions (RQs). RQ1 explores whether timing attacks strategically increases the risk of forward collisions. RQ2 tests if designing attacks with stealth in mind preserves their effectiveness despite safety interventions like AEBS or driver overrides. RQ3 compares input-level attacks to those targeting control outputs, evaluating which is more effective. The study compares several methods, including CA-Random, CA-APGD, and the proposed CA-Opt. The images show visual differences between random, gradient-based (APGD), and optimized adversarial patches. The bar chart highlights that CA-Opt consistently achieves a 100% success rate across all scenarios (SC1–SC4), significantly outperforming other baselines. The table below summarizes configurations and the number of simulations run per method. These results demonstrate that context-aware, optimized attacks on perception inputs are the most effective and resilient, even in safety-critical environments.

Simulation Methods and Results 2

This slide continues the analysis of simulation results, focusing on how adversarial patch stealthiness affects both visual detectability and attack success. The images on the left show how the patch is adaptively adjusted—shifting and enlarging in response to bounding box changes around the lead vehicle. This dynamic targeting ensures the perturbation remains effective even as vehicle positions shift. The diagram on the right reiterates the central idea: misleading the DNN into overestimating the distance to the lead vehicle, causing the ego vehicle to maintain speed or accelerate. The table presents quantitative results across varying stealth levels (λ). Even with increased stealth (smaller pixel perturbations and tighter L2 norms), the attack maintains near-100% success rates. All configurations yield high image similarity metrics (UIQ ≈ 0.993), indicating the perturbations are imperceptible to humans. These findings confirm that even highly stealthy patches can cause significant perception errors without visually alerting drivers or triggering safety systems.

Simulation Methods and Results 3

This slide analyzes the effectiveness of safety interventions under various AEBS configurations in response to different attack strategies. It reinforces the principle that human oversight is essential in Level-2 autonomous vehicles, as interventions can prevent accidents when properly engaged. The table compares the performance of CA-Random, CA-APGD, and the proposed CA-Opt method. When AEBS is uncompromised, baseline attacks are completely mitigated, while CA-Opt still succeeds nearly 49% of the time, showing its strength in bypassing safeguards. In more vulnerable setups—such as when AEBS is disabled or compromised—the CA-Opt attack achieves an 82.6% success rate, with only 17.4% of hazards being prevented, far outperforming other methods. The graph below shows that even as driver perception thresholds increase, CA-Opt maintains high success rates, while intervention rates drop, revealing a growing blind spot in human detection. This emphasizes how subtle, adaptive attacks can slip past both machine and human defenses, underscoring the urgency for stronger safeguards.

Simulation Methods and Results 4

This slide compares context-aware perception attacks (CA-Opt) to control-output attacks like StrategicOut and OptOut under varying AEBS conditions. The table on the left shows that when AEBS is fully active, StrategicOut has a low success rate (20.3%) but a high hazard prevention rate (79.7%). However, when AEBS is disabled, its success jumps to 81.9%, demonstrating how critical active safety systems are. Meanwhile, CA-Opt perception attacks strike a better balance—achieving a higher success rate (34.5%) than StrategicOut in protected environments, and maintaining stealth without needing to manipulate control outputs directly. The plots on the right provide dynamic system behavior comparisons across gas, acceleration, and speed. The vertical dashed line marks the attack initiation point. Notably, CA-Opt causes subtle, realistic deviations, while output-based attacks (e.g., MaxOut and StrategicOut) trigger more drastic changes, making them easier to detect. This underscores the advantage of input-space attacks for real-world stealth and reliability.

Real World Evaluation

This slide addresses RQ4 and RQ5, which investigate the transferability and stealth of the proposed perception attack in real-world environments. The key question is whether an adversarial patch that works in simulation remains effective when applied to a physical vehicle in a real setting, and whether it can evade modern defense mechanisms. The images demonstrate successful real-world deployment: Image 1a shows the system correctly estimating distance without the patch. Image 1b shows the patch causing a misleading prediction (i.e., the system sees the lead vehicle as further away). Image 2a–2b illustrate the real-world test platform and a collision scenario caused by the manipulated perception. Image 2c shows a benign outcome when no attack is applied. Image 2d shows the critical moment of system failure, with a brake warning following too late. These results confirm that the attack transfers well to physical settings and maintains its stealth, effectively bypassing current adversarial patch defenses and leading to dangerous consequences in autonomous operation.

Discussion

The final slide raises important open-ended questions about deploying autonomous vehicles (AVs) in real-world scenarios, especially in high-risk environments like construction zones. Citing U.S. Department of Transportation data, the slide highlights that over 42,000 crashes occur annually in work zones, with more than 800 fatalities in 2021—many involving rear-end collisions, reduced visibility, and unexpected maneuvers. These chaotic conditions are particularly challenging for both human drivers and AI systems. The discussion prompts explore: Whether these unpredictable environments could actually amplify the effectiveness or stealth of adversarial attacks. If AVs relying on DNN-based perception are less adaptable than human drivers in such dynamic settings. What additional components (e.g., driver reaction delay, AEBS effectiveness) must be realistically simulated to properly evaluate AV safety under these attack conditions. These questions underscore the need for future research that bridges simulation, real-world unpredictability, and defense robustness.

Discussion and Class Insights

Q1: How might real-world factors in construction zones increase the success rate or stealthiness of an attack on autonomous vehicles?

George: George mentioned that fully autonomous systems may struggle significantly in construction zones due to unpredictable lane changes and unfamiliar objects. He emphasized that the environment is too complex, with too many possibilities for the model to handle reliably.

Bassel & Alex : They mentioned that adding the construction sites are already tricky even for human drivers. He suggests highlighting this complexity when discussing the vulnerability of autonomous systems.

Dr. Zhou: Professor commented that the angle of the discussion is well chosen. He points out that people may ignore signs or fail to follow instructions in such zones, and having an autonomous system could actually improve safety. He reminds the class that level 2 autonomous vehicles still require human supervision and cites the importance of a final driver check before the last 10 minutes of operation.

Q2: What elements would you need to include in a simulation or field test to capture risks in construction zones (e.g., driver reaction delay, AEBS response)?

Ruslan & Sujan : Ruslan and Sujan raised the point that future simulations must include factors like driver reaction time and AEBS (Automatic Emergency Braking System) behavior. These components are essential for testing how AVs handle last-minute decisions in unpredictable construction zones.